Super-Resolution from Corneal Images

The cornea of the human eye reflects the light from a person’s environment. Modeling corneal reflections from an image of the eye enables a number of applications, including the computation of scene panorama and 3D model, together with the person’s field of view and point of gaze [4]. The obtained environment map enables general applications in vision and graphics, such as face reconstruction, relighting [3] and recognition [5]. In reality, however, even if we use a carefully-adjusted high-resolution camera in front of the eye, the quality of corneal reflections is limited due to low resolution and contrast, iris texture and geometric distortion. This paper introduces an approach to overcome these issues through a super-resolution (SR) [6] strategy for corneal imaging that reconstructs a high-resolution (HR) scene image from a series of lower resolution (LR) corneal images such as occurring in surveillance or personal videos. The process comprises (1) single image environment map recovery, (2) multiple image registration, and (3) HR image reconstruction. This is also the first non-central catadioptric approach for multiple image SR. Corneal reflection modeling. We apply a common geometric eye model, where eyeball and cornea (Figure 1 (a)) are approximated as two overlapping spherical surfaces. A simple strategy assuming weak perspective projection recovers the pose of the model by reconstructing the pose of the circular iris from its elliptical projected contour (Figure 1 (b)). A corneal image is transformed into a spherical environment map by calculating the intersection and reflection at the corneal surface. Since the eye model only approximates the true corneal geometry, it is not possible to obtain an accurate registration for the whole environment map. Instead, we assume spherical curvature for a user-defined region of interest, where we project the environment map to a local tangent plane (Figure 1 (c)). Registration further requires the forward projection from the tangent plane into the image. As common iterative methods are not feasible to handle the large number of re-projections, we apply a recent analytic method that requires solving a 4th-order polynomial equation (for the case of a spherical mirror), that is calculated in closed form [1].